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METHOD AND APPARATUS FOR CAR EQUIPPED COLOR COMPENSATION 

FOR COLOR BLINDNESS 

Technical Field 

The present invention relates generally to a method 
5 and apparatus for clearly detecting color information 
outside a vehicle and, more particularly, to an external 
observation camera for a vehicle, which compensates for the 
colors of video data based on the color contents of the 
video data input by the camera and according to the degree 
10 of the driver's color blindness, and surrounding brightness 
and chromaticity. 

Background Art 

Recently, with the development of semiconductor 
technology, inexpensive digital video cameras have 

15 appeared, and the video cameras are used as various types 
of observation cameras as Personal Computers (PCs) and the 
Internet have been popularized. From observation cameras in 
conventional buildings to web cameras in general homes, the 
use of observation cameras is increasing. In particular, 

20 digital cameras can store video frame data in a digital 
form, the image quality of the digital cameras is improved 
compared to that of conventional analog observation 
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cameras, and the prices of disks, which are storage medium, 
are falling, thus accelerating the popularization of 
digital observation cameras. 

Color blindness is a condition that occurs because 
5 cone cells in the retinas of eyes cannot perform the 
functions thereof. Most of color-blind people have 
difficulties in distinguishing or cannot distinguish the 
specific parts of the original color region visible to 
normal people. In the case where the color blindness is 
10 serious, there is the case where people cannot see colors 
at all. 

About 8% or more of the population of the world is 
color-blind people. In particular, in the case of North 
America and Europe, the color blindness is so common a 

15 handicap that the population of color-blind people is more 
than about 10% of the total population. The color blindness 
is classified into complete color blindness and partial 
color blindness, and the partial color blindness is divided 
into dichromacy and anomalous trichromacy. 

20 People with dichromacy do not have one of three cone 

cells (L, M and S cone cells) . Accordingly, people with 
dichromacy sense all the colors of a visible wavelength 
region using only two chromaticities . According to the type 
of a cone cell that is absent, the dichromacy is divided 

25 into Protanopy in the case where an L cone cell is absent, 
Deuteranopy in the case where an M cone cell is absent, and 
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Tritanopy in the case where an S cone cell is absent. 

People with anomalous trichromacy have all three cone 
cells, but one of the three cone cells functions 
abnormally. The abnormal function of the cone cell occurs 
5 because the wavelength sensitivity of the cone cell moves 
or the amount of response to the sensitivity is reduced. 
Even though the cone cell is abnormal , three cone cells are 
present, so that the people with anomalous trichromacy can 
sense most colors in a visible wavelength region. The 

10 seriousness of the anomalous trichromacy is determined 
based on the degree of the movement of the wavelength 
sensitivity and the amount of the response, and the range 
of colors indistinguishable to the people with anomalous 
trichromacy becomes wider as the degree of the anomalous 

15 trichromacy is serious. Like the dichromacy, the anomalous 
trichromacy is also divided into Protanomaly, Deuteranomaly 
and Tritanomaly according to the type of the abnormal cone 
cell . 

Most of color-blind people are people with 
20 Protanomaiy/Deuteranomaly/Protanopy/Deuteranopy, and 
account for about 90% of all color-blind people. At 
present, a method of curing color blindness does not exist, 
and the Protanomaly, Deuteranomaly, Protanopy and 
Deuteranopy have hereditary characteristics in which the 
25 Protanomaly, Deuteranomaly, Protanopy and Deuteranopy are 
inherited by children from parents, so that a constant 
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percentage of color-blind people exist. 

In the case where the color-blind people drive, they 
have difficulties distinguishing colors due to the 
surrounding environment and color blindness 
5 characteristics. In particular, Red and Green-blind people 
(dichromacy/anomalous trichromacy) have considerable 
difficulties in distinguishing the colors of traffic 
lights. Red and Green-blind people are confused by the red, 
orange, yellow and green colors of the traffic lights. 

10 Since the current traffic lights system does not provide 
any means for distinguishing the colors of the traffic 
lights to the color-blind people, the development of a 
technology that can be practically applied is acutely 
required. Furthermore, there is the added inconvenience, in 

15 which the color-blind driver cannot distinguish the colors 
of other vehicles and buildings, so that the driver cannot 
receive various related pieces of information . 

For a conventional technology that helps color-blind 
people to clearly distinguish colors, there is a method of 

20 wearing color glasses that filter out the colors of a 
specific wavelength region. When a color-blind person wears 
the color glasses and drives, the method enables the person 
to clearly distinguish colors. However, the method is 
problematic in that colors that are normally visible to the 

25 color-blind person can be differently sensed, and is 
disadvantageous in that it is difficult to consider various 
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degrees of the seriousness of the anomalous trichromacy. 

Furthermore, there is a technology that detects the 
locations of traffic lights through a satellite and 
servicing the colors of the traffic lights using 
5 characters. In this scheme, limitations in the detection of 
the accurate location of the traffic lights and operational 
accuracy in complicated areas, such as the central areas of 
towns, exist, and problems of costs and the complexity of 
the system occur. Furthermore, this scheme has a limitation 
10 in that it does not provide a measure in the case where the 
driver desires to detect the colors of preceding vehicles 
or surrounding buildings. 

Disclosure of the Invention 

Accordingly, the present invention has been made to 
15 solve the above problems, and an object of the present 
invention is to provide a method and apparatus that enables 
color-blind people, who are vehicle drivers, to sense the 
external colors of traffic lights and objects. 

Furthermore, another object of the present invention 
20 is to provide a color compensation method and apparatus 
based on the color blindness degree of the color-blind 
people, external brightness, external colors and contents 
related to the colors of external objects. 

In order to accomplish the above object, the present 
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invention allows a digital video camera to be mounted on a 
vehicle, and converts surrounding visual information into 
digital video data. Accordingly, it is possible to display 
not only traffic lights but also visual information on 
5 surrounding environment to which a color-blind driver must 
pay attention while driving. Color compensation, which is 
depending on the color blindness characteristics of the 
color-blind driver, external environment (external 
brightness and external colors) and the characteristics of 

10 a display device, is performed based on the colors of video 
data contents by analyzing the colors of the obtained 
digital video data into digital signals. 

According to an aspect of the present invention, a 
method of receiving video frame data from the external 

15 camera of a vehicle and compensating for colors for a 
color-blind person includes the steps of extracting R, G 
and B, which are color information, from the digital video 
frame data input by the external camera, measuring external 
brightness and colors from the video frame data, selecting 

20 a color range preferred by a user, compensating for colors 
by inputting the measured external brightness and colors, 
the color range preferred by the user and the driver' s 
color blindness degree, and displaying the color 
information based on user preference according to the 

25 result of the color compensation. 

Furthermore, according to another aspect of the 
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present invention, the step of selecting the user 
preference in the color compensation method is performed in 
view of the selection of the color range to be displayed, 
the magnification of interested object and the audibly 
5 * displaying of the color information. 

Furthermore, according to another aspect of the 
present invention, the step of compensating colors for the 
color-blind driver in the color compensation method is 
performed using the external brightness, the external 

10 colors, the color range preferred by the user and the 
driver's color blind degree as variables. 

Furthermore, according to another aspect of the 
present invention, an apparatus for receiving video frame 
data from an external camera of the vehicle and automatically 

15 compensating for colors for a person with anomalous 
trichromacy in a vehicle includes a means for measuring 
external brightness and colors from digital video frame data 
input from the external camera, a means for selecting a color 
range preferred by a user, a means for compensating for 

20 colors according to the input or measured driver's color 
blindness degree, external brightness, external colors and 
preferred color range, and a means for displaying the color 
information based on user preference according to the 
result of the color compensation. 

25 Furthermore, according to another aspect of the 

present invention, an external observation camera system for 
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receiving video frame data from the external camera of a 
vehicle further includes a color compensation apparatus for 
color-blind people, which includes a means for measuring 
external brightness and colors from the digital video frame 
5 data input from the external camera , a means for selecting a 
color range preferred by a user, a means for compensating for 
colors according to the input or measured driver' s color 
blindness degree, the external brightness , the external 
colors and the preferred color range, and a means for 

10 . displaying the color information based on user preference 
according to the result of the color compensation. 

As described above, color-blind drivers can accurately 
detect colors using the color compensation method and 
apparatus of the present invention when the color-blind 

15 drivers desire to sense color-related traffic signals, such 
as traffic lights, and objects outside the vehicle. 
Furthermore, according to the present invention, the color- 
blind drivers can accurately sense colors in spite of their 
various color blindness degrees and various external 

20 environments, such as external brightness and colors. 

Brief Description of the Drawings 

FIG. 1 is a view showing an example in which a color 
compensation apparatus according to the present invention 
is mounted on a vehicle; 
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FIG. 2 is a block diagram showing the configuration 
of a system for the compensation for colors according to 
the present invention; 

' FIG. 3 is a block diagram of a video frame color 
5 compensation engine according to the present invention; 

FIG, 4 is a flowchart illustrating a video frame 
color compensation process according to the present 
invention; and 

FIG. 5 is a view showing an example of an observation 
10 image including a reference image piece. 

Best Mode for Carrying Out the Invention 

An embodiment of the present invention is described 
in detail with reference to the attached drawings below. In 
the drawings, the same reference numerals are used 
15 throughout the different drawings to designate the same or 
similar components. 

FIG. 1 is a view of an example in which an external 
color observation system for color-blind drivers according 
to the present invention is mounted on a vehicle, which 
20 shows a front screen observation system. FIG. 1 shows an 
example in which the final color information is represented 
to the color-blind drivers using a monitor or speaker. 

FIG. 2 is a block diagram showing the construction of 
a system for the color compensation of an observation 
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camera for a vehicle. In FIG. 2, a camera video frame input 
unit reads video frame data from a digital camera at step 
100. At the same time, the camera video frame input unit 
receives the degree of the driver's color blindness at step 
5 200. If there is no input, the degree of the driver's color 
blindness is replaced with the previously determined degree 
of color blindness. The video frame data is input to an 
external environment condition calculating unit at step 
300, external brightness and colors are measured, and color 
10 compensation for the color-blind driver is performed in a 
color compensation unit according to a result of the 
measurement at step 500. In this case, the color 
compensation is performed in view of the range of preferred 
colors input by the driver. Finally, the color compensation 
15 based on magnification and the driver's preference is 
performed at step 400, and the video whose final color is 
compensated for is displayed on a screen at step 600. 

FIG. 3 is a block diagram showing a color 
compensation engine for color-blind drivers according to an 
20 embodiment of the present invention. Referring to FIG. 3, 
the color compensation engine is composed of six parts. 
First, Red (R) , Green (G) and Blue (B) colors are extracted 
from the input video frame data at step 510, and external 
brightness information and an external color are extracted 
25 from the input video data at steps 520 and 525. In this 
case, the external brightness is calculated from R, G and B 
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color signals that are separated from the reference image 
piece of the input video data, and the external color is 
obtained by calculating the average of R, G and B on the 
reference image piece. Thereafter, the color is compensated 
5 for to adjust it to a color range preferred by the driver 
at step 530. Thereafter, the color compensation for the 
color-blind driver is performed by inputting the external 
brightness and the external color extracted as described 
above, and the preferred color range and degree of the 
10 driver's color blindness at step 540. In this case, the 
driver's preference is considered when the color- 
compensated video is displayed at step 550 and color 
information is displayed at step 600. 

FIG. 4 is a flowchart illustrating a process of 
15 compensating for video frame colors according to an 
embodiment of the present invention. Referring to FIG. 4, 
video frame data is read by a digital camera at step 100. 
R, G and B colors are separated from the input data at step 
510. As shown in FIG. 5, a reference image piece is placed 
20 on a side or lower portion of a video screen. The external 
brightness is calculated from the colors R, G and B on the 
image piece at step 520. The external brightness calculated 
as described above is expressed by Equation 1, 



W 



= 1 

external brightness i « • i j* r 

number of pixels of reference image piece 
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ence image piece I 3 J 



(x dereference image piece I 

where ^l^^/brightness indicates external brightness specified on 

the image piece, and (x,y) indicates the location of a video 
pixel in a two-dimensional space. As shown in Equation 1, 
5 the external brightness is calculated by measuring the 
brightness on the reference image piece. Meanwhile, for 
consistent color compensation, reference external 

brightness ^| re/ere/lcel)rightness is selected and all of the color 

compensations are performed with respect to the reference 
10 brightness. The external reference brightness can be 
calculated from the external reference color. 



H_ ^| reference color ^ ^\ reference color ^\ reference color / ^ v 

^fercnee brightness """* o 



The color compensation based on the external brightness 
^l^/brightness and the reference brightness w\ rcfcfmceh ^ htncss is 

15 performed as described below at step 525. First, the 
brightness of the pixel of an input video is calculated 
using Equation 3, 



W (x, y) = R ^ + G ( X ^^ B ^ ( 3 ) 
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where (x,y) indicates the location of the pixel of an 
observation video frame. 

Thereafter, Rl, Gl and Bl signals whose brightness is 
compensated for are expressed in a homogeneous matrix as 
5 shown in Equation 4, 



~RKx,y) 
Gl(x,y) 
Bl(x 9 y) 
1 



W\ 



q | reference brightness q 



/reference brightness 

W(x,y) 

w\ 



W(x,y) 

W\ 



0 
0 



reference brightness 

W(x,y) 
0 



1 



(4) 



Msanwhile, the external color is also calculated from 
the image piece at step 525. If it is assumed that the 
average value of the R, G and B colors of the image piece 
10 is the external color, the external color is expressed by 
Equation 5, 



\ur/crao/color 



cMcma/color 



B, 



exfvnia I co\ar _ 



1 



numberof pixelsof referenceimage piece 



(x jfa-efcrencelmage piece 

Z<S(*,v) 

(jr j') e reference/mage piece 
(x^ewfcrcnceimagc piece 



(5) 



where R eslern , Uo \ m , 

G external color and 5 «/,«. a / CO ior indicate the R, G and B 
values of the external colors. For the reference, the 
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brightness of the image piece is the W\ brightncss defined in 
Equation 1. 

For consistent color compensation with respect to the 
external colors , color compensation is performed to achieve 
5 the reference color shown in Equation 2. For color 
compensation according to the reference color, the 
difference values between the external colors and the 
reference colors are compared, and R, G and B difference 
values in pixels are calculated, so that Equation 6 is 
10 obtained. 



(* ? y) — I (^| external color ^| reference color ) 

V * | externa/ brightness 

AG(X, y) = — j " — (G 1 ) Menial color ~*^| referencecolor ) 
I external brightness 

&B(X, y) = — j — (B\ exto^/coior ^ ferenccco | or ) ( 6 ) 

" | external brightness 



Thereafter, R2, G2 and B2 signals whose external 
15 colors have been compensated for are expressed in a 
homogeneous matrix as shown in Equation 7, 







"10 0 AR(x,y)" 




'm(x, y y 


G2(x,y) 




0 1 0 AG(x,y) 


X 


G\(x,y) 


B2(x,y) 




0 0 1 AB(x,y) 




B\(x,y) 


1 




0 0 0 1 




1 



(7) 



Thereafter, the ranges of colors, which are set by a 
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user, are selected and colors, which are used depending on 
the selection, are quantized as expressed by Equation 8 at 
step 530, 



G3(x y y) 
B3(x,y) 
1 



1 

Qstepjt 



1 



= round 



Qstep_G 



0 
0 



0 
0 



0 
0 

1 



0 
0 



Qstep_B 
0 1 



R2(x,y) 
G2(x,y) 
B2{x 9 y) 
' 1 



5 where , - and indicate quantization 

Qstepjl Qstep_G QstepJB 

steps for R, G and B, respectively, and round () indicates a 

function for forming an integer. The ranges of colors 

finally used are determined by controlling the quantization 

steps for the colors. If the narrow ranges of the colors 

10 are used, different colors are clearly distinguished (color 
contour) from each other, thus helping color-blind people 
to distinguish the different colors. The color 

discrimination depending on the color blindness can be 
improved through the color contour through the process of 

15 controlling the ranges of colors. 

Subsequently, driver's color blindness characteristic 
information is input at step 540. If the driver inputs new 
color blindness information at step 541, that information 
is used. If there is no input at step 542, the driver's 

20 previously used color blindness information is used. 
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The color blindness characteristic information is 
shown by Table 1. Numerical values, which indicate the 
degree of anomalous trichromacy for colors are defined as 
d R , d Q and d B with relation to Protanomaly/Protanopy, 
5 Deuteranomaly/Deuteranopy and Tritanomaly/Tritanopy, 
respectively. 



Table 1 





Color blindness features 


Medical term 


Type of color 
blindness 


Numerical expression 
of degree of color 
blindness ( d ) 


Protanomaly 


Red-Deficiency 


0.1 to 0.9 


Protanopy 


Red-Deficiency 


1.0 


Deuteranomaly 


Green-Deficiency 


0.1 to 0.9 


Deuteranopy 


Green-Deficiency 


1.0 


Tritanomaly 


Blue-Deficiency 


0.1 to 0.9 


Tritanopy 


Blue-Deficiency 


1.0 



The final color sense characteristics of the color- 
blind driver respond in combination with not only the color 

10 blindness characteristics but also the display device 
characteristics of the driver. The sense of the wavelengths 
of the colors of humans is performed by three types of cone 
eel 1 s , and the characteristics of the colors can be shown 
in L, M and S spaces. If the colors are sensed from a 

15 display device such as a monitor, the R, G and B 
characteristics of the monitor must be considered. The 
amount of colors, which can be sensed by the L, M and S 
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cone cells of humans, with respect to the characteristics 
of the wavelength of the display device is expressed by 
Equation 9, 

L R = faLWRWdA 
5 L G — \ktL(X)G(X)dX 

L B = jk / L(X)B(X)dX 
M R = jk m MWRWdA 
M Q = \k m M{X)G{X)dX 
M B = \k m M(X)B(X)dX 
10 S R = jk s S(X)R(X)dX 

Sq = \k s S(X)G(X)dX 

S B = \k s S{X)B(X)dX (9) 

In this case, if the characteristics of a color-blind 
person other than a normal person are input at steps 541 

15 and 542, the cone cells of the color-blind person respond 
differently from that of the normal person. If the color- 
blind person has Protanomaly and the degree of the 
Protanomaly is d R , the responses of the cone cells can be 
expressed using p(d R ) and q{d R ) functions. Equation 10 

20 expresses the case of Red blindness and the abnormality of 
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L cone cell. Meanwhile, the M and S cone cells are normal 
like in Equation 9, 

: z ^.b. iratea = ^p {dR )L{X-q{d R ))G{X)dX 
5 ^btahm = j p (d R W-g(d R ))B(Z)dA (10) 

where p(d R ) indicates the amount of the abnormal response of 
the L cone cell, and q(d n ) indicates the abnormal response 
to the wavelength of the L cone cell. 

Subsequently, in the case of Green blindness and the 
10 degree of thereof is d G , the response of the M cone cell is 
changed as expressed by Equation 11. The L and S cone cells 
are normal like in Equation 9, 

. M coior^ss = j p (d G )M(A - q(d G ))R {X)dk 

^blindness = \ p (d G )M (X - q{d G ))G{X)dX 
15 M «/orMmdn«s = \p(d G )M(X - q{d G ))B{X)dk (11) 



where p(d G ) indicates the amount of the abnormal response of 
the M cone cell, and q(d G ) indicates the abnormal response 
to the wavelength of the M cone cell. 

Subsequently, in the case of Blue blindness and the 
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degree of thereof is d B , the response of the S cone cell is 
changed as expressed by Equation 12. The L and M cone cells 
are normal like in Equation 9, 



i color blindness 



= \p(d B )S(A-q(d B ))R(A,)dA 
= \p{d B )S{X-q{d B ))G{X)dX 
= j p (d B )S^-q{d B ))BWdA (12) 



i color blindness 



where p(d B ) indicates the amount of the abnormal response of 
the S cone cell, and q(d B ) indicates the abnormal response 
to the wavelength of the M cone cell . 
10 In view of Equations 10 to 12, a process of 

compensating for indistinguishable colors is described 
below. First, color compensation for Red blindness, whose 
degree is d Rf is performed using Equation 13, 







color 


color 


color 
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S G 
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1 



(13) 



15 where the first IMS response matrix indicates compensation 
for color blindness, and the second matrix is a matrix for 
changing a color space from RGB to LMS. 

Subsequently, color compensation for Green blindness, 
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whose degree is d c , is performed using Equation 14, 



~R4(x,y)~ 




G4(x,y) 




B4(x,y) 
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. 1 . 



Thereafter, color compensation for Blue blindness, 
whose degree is d B , is performed using Equation 15, 



~R4(x,y)~ 
G4(x,y) 
B4(x,y) 
1 



M R M, 



color 
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color 
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color 
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L R L G L B 0 

M n M G M B 0 

£>R $G 0 
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G3(x,y) 
B3(x,y) 
1 



Final color compensation in view of external 

brightness, external colors, a user selected color range, 

and driver's color blindness information is expressed by 
Equation 16 at step 545, 
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round 
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Qstepjj 
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l 



where R(x,y) , G(x,y) and B(x,y) indicate color signals input 
by an external camera, and R4(x,y) , G4(x,y) and 54(x,j/) 
indicate the final color values obtained by compensation 
for the color blindness of the driver. Equation 16 
expresses compensation for Red blindness, and Equations 17 
and 18 express compensation for Green blindness and Blue 
blindness, respectively. 
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In Equations 16 to 18, since the case of dichromacy 
corresponds to the case where the L, M and S cone cells are 
absent, corresponding values are zero. For this 

compensation, the L, M and S are set to small specific 
constant values other than zero, and the Equations are 
calculated. 

Before the video, on which compensation for color 
blindness was performed as described above, is displayed, 
the preference of the driver related to display is 
considered at step 550. In particular, when the driver 
desires to sense the color of traffic lights or a specific 
preceding vehicle, a specific object magnification function 
of magnifying the traffic lights or preceding vehicle, and 
voice service for the selected color are included. The 
specific object magnification function is performed in such 
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a way that a driver selects an object and the object is 
magnified in a software zoom manner or in a hardware zoom 
manner at step 600. 

Furthermore, the voice service for colors is a 
5 service using a text-To-Speech (TTS) technology, and 
performed in such a way that the color of an object 
indicated by the driver is obtained from a frame screen, 
and the color is changed to characters using the TTS 
technology at step 600. 

1° The above-described embodiment is set forth to enable 

those skilled in the art to easily understand the present 
invention, but is not set forth to limit the scope of the 
present invention. Accordingly, those skilled in the art 
must appreciate the possibilities of various modifications 

15 and alternations without departing from the scope of the 
present invention. In principle, the scope of the present 
invention is determined by claims that will be described 
later. 

Industrial Applicability 

20 As described above, according to the present 

invention, a color-blind driver can accurately sense colors 
by the color compensation method and device of the present 
invention when the driver senses the colors of traffic 
signals, such as traffic lights, and objects outside a 
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vehicle. Furthermore, according to the present invention, 
the- driver can accurately determine the colors of external 
environments, such as various color blindness degrees, 
external brightness and color tone. 
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